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A  program  for  accurate  feature  detection  and  estimation  using  nonlinear  analysis 
was  completed.  The  basic  edge/feature  detector  was  originally  devised  by  Dr.  Rudin 
in  his  thesis  [1].  Improvements  using  multiscale  analysis  for  the  basic  edge  detection 
task  were  instituted  during  the  life  of  this  contract.  A  very  important  breakthrough 
came  when  feature  detection  was  combined  first  with  Cognitech’s  total  variation  based 
restoration  of  noisy  blurred  images  as  described  in  [2,3,4]  and  second  with  a  state-of-the- 
art  segmentation  device  originating  in  the  work  of  our  consultant,  Professor  J.-M.  Morel 
and  improved  significantly  at  Cognitech  by  Rudin  and  Nordby,  in  collaboration  with  our 
French  consultants  [5,6]. 

The  capabilities  of  our  total  image  processing/feature  detection  approach  can  be  best 
displayed  by  the  results  on  various  images.  Figure  la  shows  an  original  512  x  512,  256 
grey  level  “airport  image”.  Figures  (la,b)  show  conventional  edge  detectors  applied  to  this 
image.  Rudin’s  multiresolution  edge  detector  uses  rotated  masks  as  shown  in  figure  2a, 
while  figures  2(b-d)  demonstrate  the  improved  edge  detection  performance  of  this  method 
as  compared  with  Figures  l(b,c).  The  localized  masks  pick  up  more  edges,  and  thus  retains 
a  pleasing  sharpness  to  the  features.  Figures  3  and  4  demonstrate  a  similar  comparison  on 
the  “pentagon”  image. 

Of  course,  the  robust  performance  of  a  feature  detector  in  the  presence  of  noise  is 
crucial  to  its  utility.  Figure  5a  shows  the  “airport”  image  plus  additive  noise  of  standard 
deviation  15.  Figure  5b  shows  conventional  edge  detection  applied  to  that  noisy  image. 
Figures  (5c, d)  show  Rudin’s  multiscale  edge  detector  with  two  different  mask  signs  applied 
to  that  noisy  image.  Features  were  captured  with  very  little  spurious  information  retained. 

In  figure  (6a)  we  show  the  result  of  Cognitech’s  total  variation  (TV)  based  restoration 
combined  with  segmentation,  as  suggested  by  Rudin,  applied  to  figure  (5a).  Figure  (6b) 
displays  the  results  using  a  Wiener  filter  -  other  conventional  approaches  fare  as  badly. 
Figure  (6c)  shows  Rudin’s  edge  detection  applied  to  the  TV  restored  image.  It  is  interesting 
to  note  that  this  result  is  comparable  to  a  conventional  edge  detector  applied  to  the  original 
(nonnoisy,  image  (la,c)).  Figure  (6d)  shows  edge  detection  of  the  Wiener  restoration 

Figure  (7a)  shows  a  very  noisy  version  of  Figure  (3a).  Conventional  edge  detectors  give 
poor  results  -  see  (7b).  However,  Rudin’s  multiscale  edge  detection  performs  acceptably 
-  Figures  (7c,d). 

The  key  results  came  first  through  TV  restoration,  without  using  segmentation  as 
seen  in  Figure  (8a)  and  the  state-of-the-art  edge  detection  in  Figure  (8b).  These  results 
compare  to  those  in  Figure  (3c)  on  the  clean  image,  using  a  conventional  method. 

Next,  Figure  (9a)  shows  the  results  of  TV  restoration  with  segmentation  and  Figure 
(9c)  shows  the  resulting  edge  detection.  These  seem  to  give  even  better  results  than  those 
displayed  in  Figure  (3c)  on  the  clean  image.  Figure  (9b)  displays  the  poor  results  of  a 
Wiener  filter  restoration  and  Figure  (9d)  shows  edge  detection  of  that. 
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Figure  (10a)  shows  an  original  picture  of  a  chemical  plant  and  Figure  (10b)  shows  the 
edge  map  of  (10a)  using  Rudin’s  edge  detector.  Figure  (10c)  shows  multiplicative  noise  of 
standard  deviation  0.3  applied  to  this  image  and  (lOd)  displays  the  edge  map.  In  (lOe) 
and  (lOf)  we  first  restored  the  images  using  conventional  (least-squares  and  Wiener  filter) 
restoration  techniques,  while  in  Figure  lOg)  we  display  Cognitech’s  state-of-the-art  TV 
restoration  with  segmentation,  followed  by  Rudin’s  edge  detector.  Contrast  (lOg)  with 
(10e,f)  and  compare  it  to  (10b).  The  results  are  striking. 

Figure  (lla,b)  show  before:  the  “jelly-bean”  image,  then  after:  it  is  corrupted  by 
a  sizable  amount  of  multiplicative  noise.  Figure  (11c)  shows  the  restoration  using  Cog¬ 
nitech’s  TV  filters  and  (lid)  shows  the  remarkable  results  possible  using  segmentation  of 
the  original,  clean  image  -  highlights  and  edges  are  obtain  clearly.  Of  course,  this  ideal 
segmentation  is  usually  not  possible. 

In  Figure  (12b, c)  we  show  the  results  of  conventional  restoration  while  in  (13,tl-3)  we 
show  the  results  of  iterating  segmentation  and  denoising  in  Figure  (13b),  with  intermediate 
steps  shown  in  Figures  (13c,d).  As  the  iteration  proceeds,  the  restoration  approaches  the 
ideal  case  of  Figure  (lid). 

Figures  (14a,b)  show  the  before  and  after  version  of  the  clean  and  noisy  (via  multi¬ 
plicative  noise)  of  the  “chemical  plant”  image.  Figure  (14c)  shows  the  TV  based  denoising 
and  Figure  (14d)  demonstrates  this  using  the  ideal  segmentation.  Figures  (14e,g,h)  show 
various  conventional  restorations  of  Figure  (14b),  while  (14f)  provides  the  segmentation 
of  the  Wiener-filter  restored  map  at  a  chosen  scale.  Figure  (15c)  shows  the  restoration  of 
this  image  using  Cognitech’s  TV  filter  while  Figure  (15d)  shows  the  extremely  impressive 
restoration  using  TV  filter  with  segmentation.  Incredibly,  the  texture  of  the  field  in  the 
upper  right  corner  of  (15a)  is  denoised  and  restored.  We  are  currently  investigating  this 
phenomenon. 

Cognitech  achieved  great  success  in  transferring  this  technology  to  a  new  market, 
it  has  helped  to  develop  -  namely  the  field  of  criminal  justice.  As  described  in  SIAM 
News,  December  1993  issue  [7]  and  in  many  local  and  national  news  stories  our  restoration 
algorithm  was  used  to  help  identify  a  defendant  in  the  Reginald  Denny/LA  Riots  trial. 
Figure  (16a)  shows  the  original  image  taken  by  a  news  reporter  from  a  helicopter.  To 
help  identify  the  rock-throwing  suspect  we  concentrated  on  of  the  image  on  his  arm. 
Figure  (16b)  shows  an  expansion  of  this  region  using  Cognitech’s  anisotropic  diffusion 
expansion  algorithm.  Figure  (16c)  shows  segmentation  of  the  arm  (admitted  into  court 
proceedings  for  the  first  time  ever)  showing  the  suspected  tattoo.  Figure  (16d)  shows 
the  segmentation/reconstruction  of  the  arm  and  Figure  (16e)  shows  a  comparison  of  the 
reconstructed  image  and  actual  tattoo.  Identification  of  the  suspect  was  made  by  the  jury. 

Cognitech  then  moved  into  this  area  and  performed  numerous  image/video  processing 
tasks  for  criminal  justice  applications.  Figure  (17a,b)  shows  two  stills  from  a  video  of 
a  jewelry  store  robbery.  In  Figure  (17c)  we  just  interlaced  the  two  fields,  but  in  Figure 
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(17d)  we  matched  frames  using  our  new  frame  fusion  algorithm.  Figure  (17e,f)  show  the 
improvements  in  detail  via  our  frame  fusion  algorithm.  Finally  in  Figure  (17g),  we  restored 
(17f)  using  our  TV  filters.  This  evidence  was  crucial  in  the  outcome  of  that  case  -  see  [8]. 
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b)  Roberts  edge  detector 


c)  Sobel  edge  detector 


Figure  1 


b)  Rudin  edge  detector  with  mask  1x1 


c)  Rudin  edge  detector  with  mask  1  x2  d)  Rudin  edge  detector  with  mask  1  x4 


Figure  2 


b)  Roberts  edge  detector 


c)  Sobel  edge  detector 


Figure  3 


a)  Rudin  edge  detector  with  1x1  mask 


b)  Rudin  edge  detector  with  1x2  mask 


c)  Rudin  edge  detector  with  1x4  mask 


Figure  4 
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a)"Airport“  plus  additive  noise 
with  st.dev.  1 5 


b)  Roberts  edge  detector 


c)  Rudin  edge  detector  with  1x2  mask  d)  Rudin  edge  detector  with  2x3  mask 


Figure  5 
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b)  Roberts  edge  detector 


a)  "Pentagon"  plus  additive  noise  with 
st.dev.  20 


c)  Rudin  edge  detector  with  1  x2  mask  d)  Rudin  edge  detector  with  2x3  mask 


Figure  7 


b)  Edge  detector  of  TV  restoration 


Figure  8 


c)  Edge  detector  of  TV  restoration  d)  Edge  detector  of  Wiener  restoration 


Figure  9 


c)  Multiplicative  noise  with 
st.dev.  0.3 


d)  Edge  map  of  "c" 


e)  Edge  map  of  constrained 
least-square  restoration 


f)  Edge  map  of  Wiener 
filter  restoration 


g)  Edge  map  of  Cognitech’s 
T.V.  restoration  with 
segmentation 


Figure  10 


c)  Restoration  of  "b" 
using  Cognitech’s  TV 
filters 


b)  Multiplicative  noise 
with  a=  0.3 


d)  Restoration  of 
“b"  using 
segmentation  of 
original  image 


Figure  1 1 


b)  Wiener  filter 
restoration  of 
“a" 


a)  Multiplicative  noise 
with  a  =  0.3 


c)  Constrained 
least  -  square 
restoration  of 
"a" 


Figure  12 


a)  Original 


b)  Multiplicative  noise 
with  a=  0.3 


c)  Restoration  of  "b" 
using  Cognitech’s  TV 
filters 


ci)Segmentation 
of  *c" 


Restoration  using  iterative  segmentation  /  TV  denoising 


a)  Original 


b)  Multiplicative  noise  with 
a=  0.25 


c)  Restoration  of  *b"  d)  Restoration  of  "b"  using  ideal 

using  Cognitech’s  segmentation  of  original 

nolinear  filtering  image 


Figure  14 


e)  Wiener  filter  restoration  of 
•b" 


g)  Constrained  least  -  square 
restoration  of  *b" 


f)  Segmentation  of  “e 


h)  Geometrical  mean 
filter  restoration  of 
"b" 
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a)  Original 


b)  Multiplicative  noise  with 
c=  0.25 


c)  Restoration  of  "b" 
using  Cognitech’s 
T.V.  fiiter 


d)  Restoration  of  "b" 
using  Cognitech’s  T.V. 
fiiter  with  segmentation 


Figure  15 


a)  Original  Image 


b)  Expansion  of  Arm,  using  Cognitech’s 
Expansion  Algorithm 


c)  Segmentation  of  Arm,  showing  region  of 
suspected  tatoo 


Figure  1 6 


d)  Reconstruction  of  segmented  Arm 


e)  Comparison  between  reconstructed  image  and  actual  tatoo 


Figure  1 6 
(Continued) 


a)  Original  Digitized  Field  (even  lines) 


b)  Original  Digitized  Field  (odd  lines) 


z)  Expanded  Frame,  using  simple  interlacing 
of  the  above  tv\/o  fields 


d)  Expanded  Frame,  using  Cognitech’s 
Frame  Matching  Algorithm 


Figure  17 


f)  Expanded  view  of  matched  frame,  showing 
increased  detail 


g)  Expanded  view  of  matched  frame  after 
restoration  with  Cognitech’s  TV  filters. 


Figure  17 
(Continued) 


